Skip to content

feat(sleep): add DeepSeek Harness transcript source - #266

Open
Nuplum (45lab95) wants to merge 2 commits into
microsoft:mainfrom
45lab95:fix/dsh-harvest
Open

feat(sleep): add DeepSeek Harness transcript source#266
Nuplum (45lab95) wants to merge 2 commits into
microsoft:mainfrom
45lab95:fix/dsh-harvest

Conversation

@45lab95

Copy link
Copy Markdown
Contributor

feat(sleep): add DeepSeek Harness transcript source

Closes #225

Summary

Add a local, read-only DeepSeek Harness (DSH) transcript source for SkillOpt-Sleep.

The new explicit --source dsh converts DSH JSONL sessions into the existing
SessionDigest and mined-task pipeline. DSH is not added to --source auto,
and this change does not add a DSH backend, plugin, or UI integration.

Changes

  • Add a DSH JSONL session harvester supporting:
    • uncompressed session.jsonl;
    • default compressed session.jsonl.zstd;
    • concatenated Zstandard frames;
    • packed chunk records and current DSH lifecycle metadata.
  • Keep only human user requests, visible assistant text, short tool names,
    timestamps, and feedback-derived positive/negative signals.
  • Exclude reasoning, tool arguments and results, request/provider metadata,
    attachments, feedback text, injected user-role context, subagent sessions,
    and SkillOpt replay sessions.
  • Retain ordinary fork sessions without merging parent-session content.
  • Silently skip an individual session when its format or event structure cannot
    be safely understood, without blocking other sessions.
  • Add CLI, configuration, and documentation support.

Optional dependency

Add the optional extra:

dsh = ["zstandard>=0.22.0"]

zstandard is only needed to read DSH's default .jsonl.zstd session files.
It is not installed for users who do not use DSH, and uncompressed session.jsonl files remain readable without it.
Install with:

python -m pip install -e ".[dsh]"

Validation

python -m pytest tests/test_harvest_dsh.py -q
python -m ruff check skillopt_sleep/harvest_dsh.py tests/test_harvest_dsh.py
  • Local DSH Web end-to-end validation:
    • harvested 1 session -> 1 task;
    • completed the mock dry-run pipeline successfully.

@Yif-Yang

Copy link
Copy Markdown
Contributor

Thanks for the narrowly scoped, read-only importer and privacy tests. The full suite passes here with the zstandard extra (1507 passed, 9 skipped), but checking the external format contract uncovered a current-version/stale-generation blocker at 84d64abf2c3d.

Current upstream DSH (deepseek-ai/deepseek-harness at d347e703908d0406b7a7ef80e3a0e594d86b2215) declares SESSION_FORMAT_VERSION = 2. Its JSONL backend writes session.v2.jsonl[.zstd], retains immutable v0/v1 predecessors during migration, and selects the numerically highest canonical generation:

The importer only discovers session.jsonl / session.jsonl.zstd and accepts header version 0. Therefore fresh current-format sessions are silently missed. More importantly, if migration retained a v0 file next to the authoritative v2 file, this implementation successfully harvests the obsolete v0 snapshot rather than refusing the unsupported current session.

I reproduced the latter offline with the PR's valid v0 fixture plus a neighboring session.v2.jsonl: harvest_dsh() returns the old user/assistant conversation. With an explicitly version-gated reader, the safe result is to refuse that session with a useful diagnostic until its highest generation is supported, not fall back to retained history.

Please select the highest canonical generation first and fail closed on unsupported generations. Then either support the currently released logical format through a validated codec/export boundary, or explicitly pin/document this as a legacy-v0-only source with a producer version and representative fixture. Please cover fresh v2-only sessions and coexisting v0/v1/v2 generations in tests. This is a concrete format-boundary issue; no DSH execution backend or paid-model experiment is needed.

@45lab95

Copy link
Copy Markdown
Contributor Author

Thanks for review. I addressed this by supporting the current v2 logical format rather than documenting DSH as a legacy-v0-only source.

The importer now discovers canonical session.jsonl / session.vN.jsonl files, including their .zstd variants, groups retained generations by session directory, and selects the numerically highest generation before parsing. It supports v0, v1, and v2. If the selected highest generation is unsupported, it skips that session without falling back to a retained predecessor; --progress and debug logging report the selected file, highest generation, supported versions, and upgrade guidance.

For v2, the reader now validates the published header boundary, including the required isSeeded field and allowed header keys, validates consistency between isSeeded and session/end-seed lineage markers, and replays the ordered surface before deriving visible user and assistant text. It also expands the JSONL storage codec's compressed sourceEventSeqs ranges (for example, [start, end]) before validating and applying replacements. This prevents text from a shadowed pre-compaction assistant message from being exported.

I added coverage for:

  • fresh v2-only sessions;
  • coexisting retained v0/v1/v2 generations, asserting that v2 is selected;
  • an unsupported highest generation, asserting that no older generation is harvested and that a diagnostic is available;
  • v2 replacement provenance encoded as compact sequence ranges;
  • v2 header and seeded-lineage validation; and
  • a sanitized Zstandard v2 fixture generated from a DeepSeek Harness master build at d347e703908d0406b7a7ef80e3a0e594d86b2215.

The reader remains a pure, read-only Python importer: it does not start DSH, require a DSH execution backend, or make provider calls.

Validation run:

python -m ruff check skillopt_sleep/harvest_dsh.py tests/test_harvest_dsh.py
python -m pytest -q tests/test_harvest_dsh.py

70 passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Proposal: add a version-gated DeepSeek Harness transcript source to SkillOpt-Sleep

2 participants